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INTRODUCTION 



In February 1989, Statewide Assessment tests in reading, mathematics and writing were 
given to samples of eighth graders throughout Oregon. The results of the writing 
assessment have been published separately (Oregon Department of Education, 1989). 
Results and analyses of the reading and mathematics assessments ::ire presented here. 
The intended audiences for this report include the State Board, Department of Education 
staff, the Governor and legislators, local boards of education, school district 
superintendents and staff, students and their parents, business and community leaders, 
and all citizens of Oregon. These individuals all have a vital interest in the achievement 
of Oregon students. This report is intended to add another significant type of data to the 
array of information these audiences use to make decisions that affect Oregon youth. 

Although this report gives important information about the achievement of students in 
Oregon, it has several limitations. First, it focuses on eighth grade only. Until 1991, 
•.vhen an expanded statewide assessment will be given at four grade levels (3, 5, 8 and 11), 
statewide data is available for 8th grade only in reading and mathematics. 

However, several important trends based on 1985, 1987 and 1989 eighth grade 
assessments are presented along with ideas for instructional improvement. Also, the 1989 
assessment provides the first comprehensive look at student achievement in the Essential 
Learning Skills (ELS) learning strategies and basic skills that run "across the curriculum" 
(e.g., interpreting graphs and tables). The ELS goals are a vital part of the new effort to 
implement a "common curriculum" core in Oregon schools. 
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PROCEDURES 



The 1989 Eighth Grade Assessment was similar in several ways to assessments in 1985 and 
1987* As in 1987, the tests were designed to measure the Oregon Essential Learning 
Skills (ELS) adopted by the Board in December 1985. As in 1985 and 1987, test scores 
from previous years were compared using common scales in reading and mathematics 
widely used in the Northwest. Although growth in these curriculum areas was e::amined, 
several importa it changes in test content have occurred that will be noted in the report. 
The 1989 tests have increasingly expanded coverage of the ELS goals, because improved 
questions were developed to measure several of the difficult-to--measure goals. Another 
important difference between the 1985, 1987 and 1989 assessments is the date of testing. 
The 1985 test was given in April; the 1987 and 1989 tests were given in February. 

The 1989 tests represent the most comprehensive assessment to date. The 1985 and 1987 
assessments were single test-forms having 45-55 items in each of the two subject areas. 
The 1989 assessment included four test forms of 40 items each, for a total of 160 
questions in reading ana 160 questions in mathematics. Each of the forms was equated, 
based on extensive field testing and item calibration, resulting in parallel, high-reliability 
test forms in each subject area. 

This report provides an overview of the findings for the 1989 eighth grade tests. A 
supplementary report, describing test characteristics and detailed item results will be 
available as a Technical Report . 
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MATHEMATICS TEST RESULTS 
AND INTERPRETATION 



• Summary 

• Mathematics Skill Areas Tested 

• Comparison to Previous Years 

• Interpretation of the Achievement/Curriculum Scale 

• Scores by Mathematics Skill Area 

• Instructional Emphasis on Mathematics Skills 

• Oregon Compared to Second International Math Study 

• Curriculum and Instruction Concerns 
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SUMMARY 
MATHEMATICS TEST RESULTS 



• Oregon eighth grade students in 1989 had slightly lower overall mathematics 
achievement than samples collected in 1987. 

• The drop of two points in the overall mathematics score (231 to 229) from 1987 to 
1989 is equal to approximately one- third of the gain that would typically be found 
between 7th and 8th grade in Mathematics (six points). The 1989 level is still above 
the estimated national median. The drop may be partially due to the increased 
coverage of more difficult "higher level" problem-solving in the 1989 tests compared 
to 1987. 

• Oregon students show a relative increase in "higher level" math skills (e.g., 
mathematical relationships) compared to 1987 when arithmetic skills (computation) 
was the highest skill area. 

• Computation scores were down in 1989 compared to 1987. This may reflect 
inc^'eased curricular emphasis on mathematical concepts as is appropriate to the 
current "computer/information age" in American society. 

• Repeated items used in both 1987 and 1989 assessments showed performance 
improvements in Geometry but a docline in some Numeration skills. 

• Statistics and probability was an area of lower relative achievement and much lower 
instructional emphasis, especially the concepts of "median" and "range." 

• Across the five skill areas that were included in both 1987 and 1989, the proportion 
of lower achieving students (scores under 220) increased an average of 7%. 
Surprisingly, the average proportion of higher-achieving students (scores over 240) 
also increased 3.2%. This means that 1989 has brought a more heterogeneous 
mixture of achievement levels to Oregon mathematics classrooms as compared to 
1987. 

• Compared to the U.S. and 17 other countries in the Second International 
Mathematics Study , Oregon students were lower than several countries, but near the 
international median and slightly above U.S. performance levels. On 11 questions 
also given in Japan, Oregon students were 14.3% lower on the average. 
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MATHEMATICS SKILL AREAS TESTED 



The following descriptions outline the content that was assessed on the eighth grade 
mathematics tests. Each of the four test forms had the same mixture of items in each 
skill area. The content was selected from the Essential Learning Skills (ELS) adopted by 
the State Board in 1985. References to ELS numbers are provided. This list together 
with the math common curriculum goals provides a blueprint that can guide curriculum 
planning in all districts of Oregon. 

• Numeration: Read and write decimals, order fractions, decimals and percents, and 
use basic mathematical terms. (Essential Learning Skills 1.4a, b, c) 

• Computation: Solve addition, subtraction, multiplication, and division problems using 
mental, manual and calculator procedures. (Essential Learning Skills 1.4d and 1.7i) 

• Geometry: Classify and sketch or model geometric figures, and identify properties, 
such as parallel, intersecting and symmetry lines and locate points on a graph. 
(Essential Learning Skill 1.5) 

• Probability and Statistics: Solve problems using probability and statistics (including 
mean, median, mode). (Essential Learning Skills 1.6a, b, d, e) 

• Measurement: Use scale drawings to determine actual distances and directly 
measure distances and angles. (Essential Learning Skills 1.6g and 1.7e) 

• Mathematical Relationships: Solve problems using tables, graphs, formulas and 
models, including proportions. (Essential Learning Skills 1.6b, 1.6f, 6.1a and 6.3d) 

• Estimation: Estimate reasonableness of computed answers and estimate measures of 
distances and angles. (Essential Learning Skills 1.7b, e, i) 

• Problem Solving: Select relevant data, and apply logic and solution strategies to 
determine the solutions to matben^atical problems by using mental, manual or 
calculator processes. (Essential i^earning Skills 1.7a, d, and 6.3b, c, d) 
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COMPARISON TO PREVIOUS YEARS 



Overall Scores 

The distribution of total Mathematics scores for the 1985, 1987 and 1989 eighth grade 
assessments is shown in Figure 1 . 

Figure 1 

Distribution of Math Assessment Scores 



■o- 1985 
^ 1987 
Ha- 1989 



0.0 0.1 0.2 0,3 0.4 0.5 0.6 0.7 0.8 0.9 I.O 
Cumulative Proportion of Students 

The median scores (50th percentile) show a decline from 1987 to 1989, moving from 231 
to 229. The percentage of students scoring in the lower range (below 220) also increased 
from 16.8% in 1987 to 21.3% in 1989. The percentage of students scoring above 240 
decreased from 21,7% to 15.4°/ 




The drop from 233 in 1985 to 231 in 1987 is due to the difference in time of testing - 
April in 1985 versus February in 1987 — which is expected to result in a difference of 
about two points. 

Two possible explanations for the pattern of results for 1987 and 1989 in Figure 1 are: (1) 
that mathematics achievement is declining slightly, or (2) that the statewide tests have 
become more mentally challenging, producing a score distribution that is less skewed 
toward the higher end. The 1985 and 1987 tests had higher proportions of scores above 
the mean, whereas the 1989 tests had more symmetrical distributions (similar to 
bell-shaped curves). 

Two curriculum-related interpretations are possible: (1) students are more challenged by 
the "higher level" essential learning skills contained in the 1989 tests and have also shown 
a drop in computational skills, or (2) as the statewide tests align more with the ELS 
curriculum, a greater disparity is found between the state ELS goals and the instructional 
emphasis in school districts in Oregon. In other words, there may not be a full 
implementation of ELS-based mathematics instruction in Oregon schools as of the date of 
the 1989 Assessment. 
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Repeated Items 



A total of 28 items (approximately seven on each of four test forms) were repeated from 
the 1987 assessment. Few interpretaWe trends were identified at the item level, due to 
the varied nature of the results. Ten items showed strong, statistically-significant shifts 
but five were increases and five were decreases. 

Two reporting categories that did show definite trends were Geometry and Numeration. 
Six Geometry items (£LS 1.5a, c and d) were repeated and the changes in percentages of 
correct answers ranged from -4.3% to +13.5%, with an average increase of +5.7%. The 
question below (Form D, Item 9) is an example, showing an improvement of +11.7% 
(72.7% in 1987 to 84.4% in 1989). 

Form D 
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9. Point V is at (0,0), point W is at (2, 1), point X is at (4, 
2). What are the coordinates for point Y? (Use graph 
above.) 

A. (3, 6) 

B. (4, 4) 

C. (6, 4) 

D. (4, 6) 
*E. (6, 3) 

Five items in Numeration (ELS 1.4a, b and c) were in both 1987 and 1989 assessments. 
The percentages of correct answers dropped an average of -6.1% (range of differences 
was +2.4% to -10.7%). Item 31 from Form B, (see below) testing the relau 'e size of two 
fractions, decreased from 50.8% in 1987 to 40.3% in 1989. 

31 . Which of the following is TRUE ? 

2/3 < 4/5 

B. 2/3 = 4/5 

C. 2/3 > 4/5 

D. 2/3 " 4/5 

E. None of these 
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ENTERPRETATTON OF THE ACHIEVEMENT/CURRICULUM SCALE 



An achi ^ment-score scale widely used in the Northwest has been part of the Oregon 
Statewide Assessment since 1985. The scale, with numbers ranging from about 150 to 
250, is similar to standard, longitudinal scales used on other tests such as the Scholastic 
Aptitude Tests (SAT scale of 200 to 800) or the "growth" scales on publisher's 
achievement batteries. The scale, originally centered at the 5th grade average (200) by 
the Northwest Evaluation Association in the 1970's, allows comparisons across tests and 
across time in Mathematics and Reading and can be used to define curricular levels in 
these content areas. Figure 2 shows a current interpretation of the levels of the scale. 
Readers may wish to refer to this display in interpreting the findings of the 1989 
Assessment. 

Three levels of curriculum are defined in each goal area. This system is similar to the 
interpretive method used in the National Assessment of Educational Progress (NAEP) 
Curricular elements for the Essential Learning Skills (tasks or generic classes of test 
questions) are listed for ach of the three levels. The tasks repre.sent items that students 
at each level can pass with 80% probability, on the average. For example, at Level 1 in 
computation, students who score in the rarge under 220 have typically mastered (80% 
chance of passing) the multiplication of one-place decimals and whole numbers (e.g., 1.6 
times 10 equals 16). 

The use of three curricular levels will allow the statewide assessment to track the 
number and proportion of students at each level of the ELS-based curriculum from year 
to year. Also, the proportions of students at each level can be compared across goal 
areas for curriculum evaluation. 

The proportions of students at the low level of the curriculum as well as the high level 
should help in answering equity questions — Are our schools helping students at each 
curricular level? 



Figure 2 



level 1 (Under 220) 



Level 2 (220-240) 



Level 3 (Over 240) 



Numeration Recognizes that dividing a 

whole object into 6ths creates 
larger pieces than lOths do. 

Can recognize the meaning of 
a>b. 

Identifies a word-description 
of 2-place decimal numbers 
such as 208.15. 



Orders corrmon and » s- 
familiar fractions from 
largest to smallest. 

Identifies a word-description 
of 3-place decimal numbers. 



Recognizes correct or 
incorrect mathematical state- 
ments using symbols (<, =) 
separating fractions. 

Orders fractions mixed with 
Hecicnals from largest to 
smallest. 



Computation Multiplies a one-place decimal 

times whole number. 

Divides a simple food recipe 
into one-half serving. 

Computes common percentages 
(50%) or fractions (1/2) of 
whole-number amounts. 



Adds and subtracts common 
fractions and mixed numbers. 



Divides whole 
fractions. 



numbers by 



Subtracts two 2-place decimal Solves 2-step word-problems 
numbers. that require computation of 

percentages of sums of whole 
Computes less-common percent- numbers, 
agos Kit, 10%. 25%) or 

fractions (1/3, 1/8) of whole Solves percentage problems 
numbers. with an unknown (40 is 20X of 

what number?). 
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Level 1 (Under 220) 



Level 2 (220-240) 



Level 3 (Over 240) 



Identifies a line of symmetry 
drawn on a commor. figure such 
as a circle or square. 

Identifies coiTdi nates of a 
point on a gr.iph when nearby 
points are givjn. 

Di stinguishes intersecting 
versus parallel lines. 



Identifies the most likely 
outcome in a simple prob- 
ability problem (color of most 
common block in a bag of 
blocks) . 

Identifies highest or lowest 
value on a line graph or bar 
chart. 



Identifies size of an angle 
superimposed on drawing of a 
protractor. 

Directly measures length of a 
line or drawing of an object 
using a printed ruler (whole 
or common f racti on of 
measuring uni t) . 



Solves simple proportion 
problems (common ratios such 
as 3 to 4). 

Classifies simple patterns of 
integer numbers. 



Estimates length of a drawing 
of a common object given a 
printed ruler. 

Estimates length (inches, cm.) 
of common household objects. 



Reasons and solves the number 
of boxes in a 3-dimensi onal 
drawing of stacked cubes. 

Solves simple money problems 
using multiples of 5^ or 25^. 

Calculates perimeter of common 
object having equal lengths of 
sides. 



Identifies location of line 
of symmetry on a common 
household object such as a 
key or window. 

Finds a right angle embedded 
in a set of vector lines. 



Computes the probability of 
an outcome (as a fraction) 
from a spinner. 

Computes the average of a set 
of whole numbers. 



Solves proportion problems , 
wi th unf ami 1 : ^r f racti ons , 
including those in word 
probl ems . 



Estimates result of subtrac- 
tion of two decimal numbers. 

Estimates answer to division 
of two whol e numbers wi th 
remainder. 



Identifies approximate coor- 
dinates of points on a graph 
when labels are absent from 
each axis. 

Identifies irregularly shaped 
quadrilaterals. 



Distinguishes the mean from 
the median of a set of 
numbers. 

Identifies an error in an 
otherwise correct pie chart 
or graph. 



Identifies a table of numbers 
that matche*" * written des- 
cription of a 2-variable for- 
mul a. 



Estimates answer to multi- 
plication or division of two 
decimal numbers (7.90162 - 
1.9178). 

Identifies approximate answer 
to 2-step or 3-step word 
problem. 



Calculates total cost of 
retail purchase using amount 
of down-payment and monthly 
payments (whole numbers) . 

Identifies amount of savings 
from purchase of i tems on 
sale versus regular price. 

States whether multiplication 
or di^'ision is needed to 
solve a problem. 



Solves more complex 2 or 
3-step problems such as 
calculating the arrival time 
for a trip by car given time, 
distance and average miles 
per hour. 

Finds perimeter of irregular 
figure where the length of 
certain segments must be 
inferred . 



Di recti y measures 1 i ne or 
object drawn next to printed 
ruler (to nearest 1/2, 1/4 or 
1/8) inch). 

Di scrimi nates between angl es 
of di f f erent si zes (drawn 
over printed protractor). 



Uses proportionality to 
determine height of object in 
drawing (where sides of 
superimposed right triangle 
are given) . 

Computes volume of a cylinder 
using formula. 



Identifies mi:ising value in a 
2-variable table based on a 
formula (d=4t). 



Identifies a graph that 
illustrates a 2-variab':e 
relationship. 



SCORES BY MATHEMATICS SKILL AREA 



In 1989, eight skill areas were tested, as described on page 8. Four of these areas are 
similar to categories used in 1987 (Computation, Estimation, Problem Solving and 
Mathematical Relationships). Geometry and Measurement were merged in 1987 but are 
reported separately in 1989. Figure 3 shows the results from 1987 and 1989, using the 
median achievement scale scores for each skill area and for the overall total scores. 



Figure 3 



Math Subtest Scores, 1987 and 1989 



236 




222 H 1 1 1 1 i i \ 1 

Comp- Estim. Geom. Meas. Prob Math Num- Stat./Prob. Total 
utation Solv. Rel. eration 

Performance in 1989 was lower in Computation (Arithmetic Skills in 1987) but higher in 
Mathematical Relationships. Performance levels were particularly lower in Statistics and 
Probability, a new reporting category for 1989. Inspection of individual items indicated a 
consistent finding that students were unfamiliar with the concept of "median" (as in 
"Given this list of annual salaries, what is the median salary?"), which was interpreted 
incorrectly as an average (mean) rather than the "middle value." 

Although o3rformance levels were slightly higher in Estimation, Geometry, Measurement 
and Problem Solving, the differences were very small. Problem Solving remains one of 
the lower performance areas across all skills. The increase in Mathematical 
Relativ~)nships is encouraging, however, since it involves graphs, tables, proportions, and 
formulas that often include "higher level" mathematical thinking. 
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The proportions of students at each curricular level in each okill area are shown in Figure 
4. Most of th3 skill areas show the trend toward a decrease in the proportion of students 
In Level 2. This means that there are more students ir Level 1 and in Level 3 in 1989 as 
compared to 1987. In other words, there was an increase in both the less-able and the 
more-able students. Two exceptions were Computat on and Problem Solving where Level 
1 increased but the higher level decreased in 1989. (Note. Although the medians for 
Pioblem Solving were 227 in 1987 and 228 in 1989, the average scores were reversed, 
230.3 in 1987 and 228.4 in 1989, t cause the 1 87 dstribution was skewed toward the 
upper end). 

Figure 4 



PROPORTION OF STUDENTS 
AT THREE ACHlEVEMErn" LEVELS 
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INSTRUCTIONAL EMPHASIS ON 
MATHEMATICS SKILLS 1987 to 1989 



A questionnaire was distributed and completed by a sampling of eighth grade 
mathematics teachers during the time of the statewide assessment in February, 1987 and 
1989. For each of the skill areas on the test, they indicated the amount of instructional 
time they devoted to each skih in their classrooms as shown below: 

Each question looked lik? the following: 

Briefly Developed or No 

Goal Reviewed Introduced Expanded Instruction 

1. Computation 

2. Estimation 



Percentages of teachers who provided "expanded instruction" beyond a review or 
introductory level are shown below for 1987 and 1989. 



Percent of Teachers 
Providing Expanded Instruction 



Skill Area 


1987 


1989 


Computation 


58% 


56% 


Estimation 


32% 


28% 


Geometry* 


29% 


34% 


Measurement* 


29% 


21% 


Problem Solving 


55% 


48% 


Math Relationships 
Numeration* 


27% 


27% 




43% 


Statistics/Probability* 




7% 



*Note: Geometry* and Measurement were 
combined in 1987. Numeration and Statistics 
were not separate skill areas in 1987. 

As expected from the test score results, Statistics and Probability was the area of lowest 
instructional emphasis in 1989. Most of the other skill areas for 1989 are surprisingly 
similar to the 1987 percentages, given the shifts in Computation and Math Relationship 
scores. Problem Solving and Computation continue to be the two highest areas of 
instructional emphasis. 
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OREGON COMPARED TO THE SECOND 
INTERNATIONAL MATHEMATICS STUDY (SIMS) 



The 1989 eighth grade assessment included a total of 15 items from the Second 
International Mathematics Study (SIMS) sponsored by the International Association for the 
Evaluation of Educational Achievement (Travers, et al., 1986). Data from samples of 
13-year-old students (8th grade in the U.S.) in the U.S. and 17 foreign countries can be 
compared to the results in Oregon. Comparisons were made using the percentage of 
students answering each item correctly. Table 1 shows the results for each SIMS item. 

In comparison to students in other countries, Oregon students performed slightly above 
the international median (percent correct) on the average. Oregon was higher than the 
international median in Estimation, Geometry and Problem Solving, but lower on 
Measurement and Statistics items. The higher scoring countries were Japan and the 
Netherlands, which were occasionally more than 20 percentage points higher on individual 
items than Oregon or the USA. The United States was near (slightly below) the 
international median on the average. The lowest scoring countries (averaging 20 
percentage points lower) were small developing countries in Africa and the Far East. 



U.S. Comparison 



In comparison to the USA, Oregon averaged 4.9% higher across the 14 SIMS items in the 
1989 assessment. Of the 14 Oregon/U.S. comparisons, (see the right-hand column of 
Table 1) nine were significantly higher for Oregon, and only two were significantly lower 
(Items 077 in Numeration and 163 in Statistics). Oregon was higher than the U.S. in 
Computation, Estimation, Geometry, Measurement, and Problem Solving. Oregon was 
lower on the one Statistics item included (SIMS 163). 

The skill area showing the greatest increment was Geometry (four items in ELS 1.5) in 
which Oregon averaged 8.8% higher than the USA sample. Item 031 shown below, is an 
example of a Geometry item (ELS 1.5b) on which Oregon students did well (Oregon 71.6%, 
USA 59.0%, Japan 81.6%). 



SIMS No. 031 



S 

Q R 



T 

U V 



M N O 
Z 



X w 



The diagram shows a cardboard cube which has been 
cut along some edges and folded out flat. If it is 
folded to again make the cube, which two corners will 
touch corner P? 



A. corners Q and S 

B. corners T and Y 

C. corners W and Y 
*D. corners T and V 

E. corners U and Y 
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Table 1 



Percent-Correct Results for Items from the Second International Mathematics Study (SIMS) 



Report 


SIMS 
No . 


Inter- 
national 
Medi an 


Japan 
Resul ts 


U.S. 
Resul ts 


1989 
Oregon 
Resul ts 


Oregon Compared to 
Intl. 

Median Japan U.S. 


ComDutati on 
(1.4d) 
(^/i) 


044 
110 


56.0 
43.1 


68.9 
55.8 


45.7 
42.3 


52.7 
46.4 


-3.3 
+3.3 


-16.2* 
-9.4* 


+7.0* 
+4.1 


Estimati on 

(1.7t) 
(1.7b) 


045 
185 


26.9 


33.9 


27.4 
63.8 


37.4 
70.3 


+10.5* 


+3.5 


+10.0* 
+6.5* 


Geometrv 

(1.5b) 
(1.5d) 
(1 .5a) 
(1.5b) 


031 
126 
159 
198 


60.3 
32.8 

en T 

60 .7 


81.6 
63.0 
87 


59.0 
30.6 

CO 0 

33.6 


71.6 
35.3 

71 1 
1 I .0 

39.6 


+11.3* 
+2.5 
+10.6* 


-10.0* 
-27.7* 
-15.9* 


+12.6* 
+4.7* 

+12.1^ 
+6.0* 


Measurement 
(1.6g) 


164 


67.5 


88.2 


51.8 


58.2 


-9.3* 


-30.0* 


+6.4* 


Problem Solving 
(1.7a) 


141 


58.5 


63 .3 


48 .8 


04, b 


+6.0* 


+1.2 


+15.7* 


Math Relations 
(1.6g) 


197 






41.1 


40.3 






-0.8 


Numeration 

(1.4a) 
{1.5a) 


007 
077 


52.4 
64.7 


70.0 
76.4 


53.9 
59.0 


55.4 
52.1 


+3.0 
-12.6* 


-14.6* 
-24.3* 


+1.5 
-6.9* 


Statistics 

(1.6a) 


163 


39.8 


45.7 


42.4 


31.8 


-8.0* 


-13.9* 


-i0.6* 



* Significant difference between proportions, p<.001 

Note. SIMS percentages estimated for Pebruary (64% of U.S. Pre/Post interval). 
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An example of an item that showed lower performance in Oregon (52.1%) compared to 
the USA (59.0%) and Japan (76.4%) is in Numeration (ELS 1.4A), Item 077 shown below. 



SIMS No. 077 



1,0 



1.1 



The position on the scale indicated by the arrow is 



A. 
*B. 
C. 
D. 
E. 



1.004 

1.04 

1.08 

1.4 

1.8 



Other items with large Oregon-USA differences included Item 141, a Problem Solving 
item showing a higher percentage correct (+15.7%) in Oregon (finding how long it takes 
the sound of a car horn to reach you if the car is 714 meters away and the speed of sound 
is 340 meters per second). 

Item 045, in Estimation, was also higher (+10.0% above the USA) in Oregon 
(approximating the value of 0.2131 times 0.02958). Geometry Item 159 was also higher 
(+12.1%) in Oregon (identifying the top view of a cube that has one corner cut off). 
Finally, Item 163 in Probability showed lower performance (-10.6%) in Oregon (finding 
the probability of drawing a red button from a jar containing one red and five black 
buttons). 

Comparison to Japan 

Japan averaged 14.3% higher than Oregon on the 11 items in Table 1 given in all 
locations. Of these 11 items, nine were significantly lower for Oregon and only two were 
slightly higher than Japan. Oregon compared more favorably to Japan in Estimation and 
Problem Solving. Results for Oregon students in Computation, Measurement, Numeration 
and Statistics were particularly low compared to Japan. In Geometry, Oregon students 
were significantly lower than Japan, but higher than the U.S. results. 

The critique of 8th-grade mathematics curricula in the United States by Travers, et al. 
(1986) was that "arithmetic appears to dominate ... In many countries, students at this 
level have moved on to more ambitious treatment of algebra and geometry . . . 
probability and statistics." This appears to be less characteristic of Oregon, where 
Geometry, at least, appears to be more emphasized. 
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CURRICULUM AND INSTRUCTION CONCERNS 



In preparation for the 1991 assessment, continued and expanded dissemination of 
information on Essential Learning Skills within Oregon districis is recommended. 

Sharing instructional strategies and materials for ELS's in Problem Solving, 
Numeration, and Statistics/Probability may be particularly important. Sharing 
among teachers and between schools and districts should be encouraged. 

Curriculum self-studies and dialogue among Oregon educators are recommended on 
the question of the relative weighting of instructional time on Computation versus 
the other reporting categories. Research in uther states has indicated that teachers 
often continue to emphasize computation (tiie Second International study also 
highlighted this concern) at a time when the computer 'ige emphasizes problem 
solving and mathematical thinking. 

Inservice workshops and other staff deveiopn^ent activities should focus on ways to 
adjust curriculum and instruction toward p;c ic:^ r.olving, mathematical relationships 
(graphs, tables, two-variable formulas) and esiiiUuon to respond to future demands 
of the information age. 

If Oregon students are to be prepared for the competitive nature of the pacific-rim 
economy, new ways need to be explored for reducing the "gap" betw en Oregon and 
Japanese performance in areas such as Measurement, Geometr> and Statistics at the 
8th grade level. 
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READING TEST RESULTS 
AND INTERPRETATION 



• Summary 

• Reading Skill Areas Tested 

• Comparison to Previous Years 

• Interpretation of the Achievement/Curriculum Scale 

• Scores by Reading Skill Area 

• Instructional Emphasis on Reading Skills 

• Success of the Complete-Passage Method 

• Curriculum and Instruction Recommendations 

• APPENDIX: Sample Complete Passage 
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SUMMARY 



Reading Test Results 



Oregon eighth grade students had median Reading scores that matched the 1987 
median. 

Significant gains were made from 1987 to 1989 in three areas — Literal Understanding 
(ELS 2.1), Locating Information (using dictionaries, tables of content) and Implied 
Understanding (ELS 3.1). 

An important curricular trend was shown in the increased instructional time by 
teachers in the area of Evaluative Understanding (ELS 6.1, 6.2 and 6.4). This goal 
area requires students to think analytically and use criteria for judging the quality of 
what they read. However, the new emphasis reported by teachers did not result in 
improved test performance in 1989. The median Evaluative Understanding score 
moved from 227 in 1987 to 222 in 1989 and the proportion of students scoring over 
235 decreased from 37.8% to 17.2%. Perhaps the new instructional emphasis is not 
specifically targeted on the ELS goals in this area. 

The skill area of Mass Media (Accuracy of Information) showed a decrease from 1987 
levels and little change was found in instructional emphasis on Mass Media by 8th 
grade teachers. 

Use of a "complete story" passage in the 1989 test was effective. This innovative 
mode allows educators to assess student comprehension and reading strategies on a 
test that is more similar to "real life" than traditional "short passage" or 
sentence-reading methods of testing. 
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READING SKILL AREAS TESTED 



The following descriptions outline the content and E^>sential Learning Skills assessed on 
the eighth grade reading tests. Each of four test forms had a mixture of items in each 
skill area, and the forms were equated and comparable in difficulty. This list provides a 
blueprint of the reporting categories to be used in the 1991 statewide assessment. 

• Word Meanings: Use context clues, dictionaries, and glossaries to find word 
meanings. (Essential Learning Skill 1.2.) 

• Literal Understanding: Recall details of a passage, and distinguish facts and 
opinions. (Essential Learning Skill 2.1.) In 1987 this included "main ideas." 

• Locating Information: Use different parts of a book (e.g., table of contents, index, 
illustrations, etc.) to locate information. (Essential Learning Skill 2.2.) 

• Implied Understanding: Draw logical conclusions, recognize cause and effect, make 
inferences, and predict probable future actions. (Essential Learning Skill 3.1.) 

• Mass Media: Separate relevant and irrelevant information. Identify propaganda 
techniques, biases and stereotypes, and evaluate mass media influences. (Essential 
Learning Skills 4.1 and 4.4.) In 1987, this emphasized "accuracy of information." 

» Evaluative Understanding: Understand metaphors, evaluate information related to 
hypotheses, and identify criteria for making evaluations. (Essential Learning Skills 
6.1, 6.2 and 6.4.) In 1987, the emphasis was on 6.4 only. 



Or 
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COMPARISON TO PREVIOUS YEARS 



Total Reading Scores 

The distribution of total reading scores for the 1985, 1987 and 1989 eighth grade 
assessments is shown in Figure 5. 

Figure 5 

Distribution of Reading Assessment Scores 
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250 



240 



230 



220 



210 



200 






1985 




1987 




1989 



Cumulative Proportion of Students 



The 1985 results are uniformly higher, because the testing occurred in April as compared 
to February in 1987 and 1989. The expected difference of two scale points between 
February and April is present. Also, the 1987 and 1989 tests included more ELS-based 
items. The 1985 test included an entire block of literal comprehension questions (from 
the Degrees of Reading Power tests). 
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The median total scores for 1987 and 1989 were both 225. However, the percentage of 
students with lower scores (below 215, Basic level) was slightly greater in 1989, and the 
percentage of students with higher scores (above 235, Advanced level) was significantly 
greater. This suggests a slight growth in the "gap" between less-able and more-able 
readers. The two pie charts below show the change in proportions of students from 1987 
to 1989. 



Proportions of Students 
at Three Reading Skill Levels 




The total reading score can be converted to a nationally-recognized scale from the 
Degrees of Reading Power (DRP) tests The DRP, formerly distributed by the College 
Board, assesses literal comprehension of passages (using the "fill-in-the-word" or "cloze" 
procedure) and predicts the readability of commonly used textbooks, newspapers and 
other reading material Students obtain DRP scores and text books also receive DRP 
ratings on the same numerical scale. The 1989 median is approximately equal to a DRP 
value of 65, 5, This value is substantially above the level of the Oregon Driver's manual 
(DRP 58), a booklet often read in the 8th or 9th grades- The value is above adult sports 
magazines (DRP 62) and just below the calculated average for Oregon newspaper 
front-page stories and editorials (DRP 66 to 69), This means th' the majority of Oregon 
8 th graders have good comprehension of age-appropriate and recreational reading but 
would be slightly challenged by newspaper front-page stories and editorials. About 7% of 
Oregon 8th-grade students are at or below the level of children's fiction (DRP 48). There 
are also about 15% of 8th graders who are leading at or above 'he level of advanced 
professional journals (DRP 77). 



Repeated Items 

A total of 28 items (approximately seven items per form) were repeated exactly from the 
1987 test. The results were a mixture of gains and losses, with few interpretable trends. 



ERLC 



23 



One area that showed a clear increase on all five of the repeated items in its category 
was Literal Understanding. These items increased an average of 7.1% correct answers. 

The category of items that showed the largest gains was "fact versus opinion" (ELS 2.1a). 
The item below, which showed a gain of 8.9%, is an example of this trend. 



New reports on T.V. should always show both sides of a story. I think that we 
often don't get a fair view of events. The best way to give the news is to go 
into detail on each story. 



Which statement is true about the paragraph? 

1 . There are no opinions expressed. 
74.0% *2. The whole paragraph is opinions. 

3. Only one sentence expresses an opinion. 

4. Two of the three sentences express opinions. 

(65.1% in 1987) 
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INTERPRETATION OF THE ACHIEVEMENT/CURRICULUM SCALE 

IN READING 



An achievement scale similar to the one used in Mathematics has also been used in 
Reading since 1985. The scale allows comparisons among skill areas and across time. 
The scale was originally centered at the 5th grade average (200) in the 1970's. Figure 6 
shows a current interpretation of the levels of the scale. Readers may wish to refer back 
to this scale while reading subsequent sections of the report. 



Figure 6 

READING SCALE 



240 



230 



220 



210 - 



200 



190 - 



Professional Journals* 
Firsi-year College Textbooks 

Average Oregon Newspaper Front Page 

1989 OREGON 8th GRADE MEDIAN 

Sports Magazines 

Oregon Drivers Manual 

Spring 8th Grade National Median (1930)** 



Children's Fiction Magazines 

Spring 5th Grade National Median (1980) 



^oring 3rd Grade National Median (1980) 



*For each type of reading material, the scale value is the average readability 
(instructional) level from the Degrees of Reading Power , a test of literal comprehension 
using the cloze procedure that predicts the readability of text books and other material. 

**Grade levels are national medians from an equating study by Holmes (1980). 
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CURRICUI.AR LEVELS OF THE READING SCALE 



The reading scale can also be divided into three levels representing increasingly 
more difficult reading tasks. This system is similar to the reporting method used 
in the National Assessment o f Educational Progress (NAEP). Curricular elements 
derived from test items measuring the Essential Learning Skills (ELS) are listed 
for each level in each goal area (see Figure 7). The elements at each level are 
those that students who score in that range (e.g., 215-235 for Level 2) have 
typically mastered. Mastery is defined as an 80% chance of passing the item, on 
the average. For example, a student who scores around 225 will typically pass 
items dealing with grade-level suffixes (in Word Meaning). 

The proportion of Oregon students at each level, in 1989 and subsequent years, 
should provide useful information for curriculum planning. Effects on students at 
the low, average, and high levels of the curriculum for grade 8 can be traced over 
time. 



Figure 7 



Level 1 (Under 215) 



Level 2 (215-235) 



Level 3 (Over 235) 



Word Meaning 



Recognizes the meaning of Recognizes the meaning of Recognizes the meaning of 

words in context (6th to 7th words in context (8th to 10th words in context (11th to 

grade vocabulary). grade vocabulary). I2th grade vocabulary). 

Recognizes the meaning of a Identifies meaning of grade- Identifies the meaning of 

word with a common prefix level suffixes (rash-ness) . words with uncommon suffixes 

(rg-stated). (assur-ance) . 



Literal 
Understanding 



Identifies clear, brief state- 
ments of opinion. 



Identifies correct outline of 
a science passage. 



Arranges 3 events from a brief Arranges 4 
fiction passage in order of literary 



time. 

Recalls 
a story. 



main factual theme of 



events from a 
passage or 3 
elements of a poem in order. 



Identifies time order of one 
event in a sequence embedded 
in a fiction passage. 



Identifies staterrients that 
are n^ facts or not included 
in a technical passage. 

Arranges sequence of facts 
from a technical (science, 
social studies) passage. 



Locating Identifies the page location 

Information of a topic in a simple Table 
of Contents or Index. 

Identifies the correct verbal 
statement of an important fact 
shown in a line graph. 



Recognizes a statement that Identifies diacritical mark- 
cannot be supported by a line in.\^s for pronouncing words, 
graph or pie chart. 



identifies a major conclusion 
implied in a line graph of 
two variables. 



Summarizes the underlying 
th3me in a section from a 
Table of Contents. 

Identifies the rate of change 
shown In a graph and i ts 
description. 
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Uvel 1 (Under 215) 



Level 2 (215-235) 



Level 3 (Over 235) 



Implied 
Understanding 



Identifies implied facts or 
background infonnation in a 
short poem. 

Identif i es an obvi ous con- 
clusion (unstated) from a 
passage. 



Identifies 
in a short 



the mood expressed 
poem or passage. 



Recognizes simple cause-and- 
effect or predictions of 
future actions from a 
factual, nonfiction passage. 



Recogni zes metaphors in 
fiction passages or poetry. 

Identifies questions that 
would lead a reader to deeper 
meanings of a poem. 

Recognizes descriptions of 
authors purpose in an 
advanced expository passage. 



Mass Media Identifies obvious bias or 

common stereotyping present 
in a short passage. 

Identifies examples of influ- 
ence Of media on teenage 
retail purchases. 



Di scri mi nates among brief 
statements that are more 
or less bel i evabl e. 

Verifies whether a statement 
accurately reflects the in- 
formation in a bar chart or 
graph. 



Identifies the information 
that would not be useful in 
finding answers to an inves- 
tigative research inquiry. 

Identifies examples of subtle 
bi as , uncoawon stereotypi ng, 
subtle propaganda and adult- 
oriented influences of media 
on society. 



Evaluative 
Understanding 



Identifies well-known criteria 
for evaluating media perform- 
ances (TV, recorded music). 

Identif'es clear statements of 
evidence in passage that can 
be proven true or false. 



Identifies statements of 
conclusions that are based 
on opinions rather than 
evidence in a passage. 

Identifies similarities be- 
tween points of view of two 
characters in a story. 



Identifies statements of 
societal criteria for eval- 
uating artistic performances 
or works of art. 

Recognizes when stated 
conclusions are based on 
evidence presented in a 
detailed table, graph or 
di agram. 
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SCORES BY READING SKILL AREA 



In 1989, the six skill areas assessed were very similar to the six areas assessed in 1987, 
although Literal Understanding was called "Main Ideas, Facts & Opinions" in 1987 and 
Mass Media was called "Accuracy of Information." More items specifically aimed at 
mass-media influences (ELS 4.4a and 4.4b) were added In 1989. 

Figure 8 shows the median scores in each skill area and total scores for 1987 and 1989. 
This profile represents the "middle student" in each of the areas. The most significant 
changes are in Literal and Implied Understanding. The movement toward curricular 
emphasis on "higher level" thinking and interpretive reading may be reflected in the gain 
for Implied Understanding (ELS 3.1) and for the "fact versus opinion" items on the test 
(ELS 2.1a). 

The 1989 profiie of scores also indicates more balance of curricular /instructional 
coverage since all the skill areas are more similar in level. The next section. 
Instructional Emphasis, seems to indicate some of this shift as well. 



Figure 8 



Reading Subtest Scores, 1987 and 1989 




230 
225 
220 
215 
210 
205 
200 
195 
190 



* 1989-Median 
<y 1987-Median 



Word Literal Locating Implied Mass Eval. 
Meaning Underst. Inform. Undorst. Media Underst. 



Total 



It is somewhat surprising that Evaluative Understanding showed a decline from 1987. 
However, the area was called "Reasoned Evaluations" in the 1987 test because it 
concentrated on ELS 6.4. The 1989 tests sampled more types of items including those in 
classifying ideas (ELS 6.1a), evaluating metaphors (ELS 6.1b), interpreting differences 
between two explanations (ELS 6.2a) and identifying hypotheses (ELS 6.2b). Also, further 
investigation of the scores showed that the percentage of students at the higher levels 
(above scores of 235) decreased from 37.8% in 1987 to 17.2% in 1989. The percentage of 
students at the lower level (below 215) also increased from 22.8% to 25.7%. This means 
that the more complex Evaluative Understanding category in 1989 revealed a greater 
proportion of average-level readers at 8th grade than was seen in 1987. 
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The Mass Media category also had new types of items in 1989. Items on stereotyping and 
bias (ELS 4.1c) were greatly expanded in 1989 vind additional items evaluating mass media 
influences on society (4.4a) were included. An example of an item on which performance 
was below average is shown below. 

On March 25, 1911, a fire broke out in the Triangle Shirtwaist Company in 
which 146 women workers were killed. There were no sprinkler systems, only 
one fire escape, and factory doors were kept locked. A jury acquitted the 
owners because as one of the jurors said. "I think the girls who work in 
factories are not as intelligent as tl ose in othor walks of life and were 
therefore the more susceptible to p^nic." Vvhat bias dees this person 
demonstrate? 



1. 

49.6% *2. 

3. 
4. 



Factory owners are not responsible for maintaining a safe place to work. 
Female factory workers are not capable of thinking for themselves. 
Workers are responsible for their own working conditions. 
No one is responsible for accidents. 



Mass Media 
ELS 4.1c 

Proportions of students at each curricular level of each skill area in reading are shown in 
Figure 9. 

Figure 9 

PROPORTION OF STUDENTS 
AT THREE ACHIEVEMENT LEVELS 
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Results for 1989 can be compared to 19S7 in each area in Figure 7. The size of the 
"bands" for Level 1 (low) and Level 3 (high) are particularly meaningful. Substantial 
decreases in the proportion of Level 1 students were found in Literal Understanding, 
Locating Information and Implied Understanding. These three areas, and Mass Media, 
also show increases in the proportion of Lew;: 3 students, with Implied Understanding 
having a major increase (from 0% in 1987 to 1< \% in 1989). Evaluative Understanding 
showed a drop from 1989 at Level 3, again due, perhaps, to the increased complexity of 
items in this area. However, it should be noted that a large number of students in 1987 
were in the range of 236-237 (22.6%). When students above 237 were compared in 
Evaluative Understanding, the results were more similar across the two years (15.2% in 
1987 to 12.7% in 1989). 
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INSTRUCTIONAL EMPHASIS ON 
READING SKILLS 1987 to 1989 



A questionnaire, similar to the one used in 1987, was completed by a sampling of eighth 
grade teachers who administered the reading tests. The questionnaire asked them to 
indicate the amount of instruction their students received on each of the curriculum 
areas of the test. 

Each question looked like the following: 

Britifly Developed or No 

Goal Reviewed Introduced Expanded In^trupt^Q" 

1. Word Meaning 

2. Literal Understanding 

The percentages of teachers who indicated "Developed/Expanded" instruction (beyond a 
review or introductory level) or "No Instruction" were particularly noteworthy. The 
results for both 1987 and 1989 are shown below. 



Percent of Teachers 
Providing "Expanded" or No Instruction 



Skill Area 

Word Meaning 
Literal Understanding 
Locating Information 
Implied Understanding 
Mass Media/ Accuracy 
Evaluative Understanding 



Expanded 


No Instruction 


1987 1989 


1987 


1989 


51% 55% 


14% 


3% 


53% 58% 


15% 


5% 


39% 37% 


16% 


12% 


45% 47% 


16% 


8% 


17% 17% 


27% 


34% 


10% 35% 


42% 


11% 



The largest increase in instructional emphasis is on Evaluative understanding for which 
35% of teachers now report expanded coverage (up from 10% in 1987). Also, the "No 
instruction" categories showed significant declines, meaning that more elements have 
entered the classroom since 1987. 

Except for decreases in the percentages of "no instruction," the first four areas— word 
meaning, literal understanding, locating information, and implied understanding— had 
nearly the same coverage in 1987 and 1989. The two highest areas continue to be basic 
instruction in word meanings and literal understandings. 

The dramatic increase in coverage of Evaluative Understanding and the smaller increase 
in Implied Understanding are very positive trends that should help students prepare for 
the "information age." As was shown in the previous section on Reading Skill areas, the 
instructional emphasis on evaluative reading may have been targeted more to the average 
readers (a larger group in 1989). Alternatively, the effects of instruction may be delayed 
or targeted at facets of Evaluative Understanding that are different from the categories 
of the Essential Learning Skills. Also, new emphasis may have resulted, indirectly, m the 
gains for Implied Understanding. 
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SUCCESS OF THE COMPLETE-PASSAGE METHOD 



An innovation in Reading assessment was included in one of the 1989 test forms -- a 
complete short-story. The Flying M?}chinft by Roy Bradbury was included in three pages of 
the test booklet. A pre-reading section with a concept mapping of the story was given 
first so that passage familiarity" and background information on ancient China could be 
controlled. The story and questions are included in the Appendix, which can bvi used as a 
mode for classroom or district testing. A total of 21 questions were included covering 
the skill areas (except Locating Information and Mass Media), ard they also provide 
models of ELS-related questions. 

Several sources of evidence show that this method of testing was successful: 

• About 90% of the students correctly identified the background information taught in 
the pre-reading section, indicating that the passage was equitable to students in 
terms of familiarity. 

• Most of the Literal and Implied questions showed approximately 80% correct 
answers, indicating that the longer passage was fair. 

• Data across test forms indicated that, with scalii the forms were comparable and 
did not put the students given the long passage at a . ,sadvantage. 

T^^ only improvement that was suggested by the results was to format the Word Meaning 
Items differently. Apparently, students could not easily find the target words in the 
passage (although they were underiined) to obtain "context clues" since the 
percent-correct results were nn the low side (35% to 58%). Better markings for each 
word (e.g., with a graphic symbol) or repetition of the target sentence may be required. 

The pre-reading section also provides a model for classroom instruction and assessment 
A concept-map method is used to assure that stijdents are familiar with Ancient China 
and Its Emperors, before reading the story. Ot>'er methods or diagrams useful in 
pre-reading would include advanced organizers, clusters, semantic webs, now charts or 
sunbursts. Each of these helps the student to plan reading strategies and to actively 
search for key elements in the passage. 

The method is strongly recommended for classroom assessment because it is closer to 
real life reading of complete books and articles. If specific words are tested a 
repeating of the context sentence is recommended. 
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CURRICULUM AND INSTRUCTION CONCERNS 



• The trend toward increased emphasis on implied and evaluative understanding in 
reading should continue as a curricular priority in Oregon. 

• Searches for innovative ways to reach the more-able reader should be continued, 
given the trend toward a higher percentage of students in this category. 

• Studies of the background characteristics of students in both the less-able and 
advanced groups should be encouraged so that targeted interventions or 
enhancements can be appropriately designed for these expanding groups. 

• English language arts teachers should examine the category of Evaluative 
Understanding and its ELS components, to see if new instructional emphasis includes 
these goals. 

• The skill areas in Mass Media continue to be challenging, and will require 
coordination between teachers from the variety of 8th grade courses in which these 
skills are taught. 

• The innovative use of "complete passages," whether they are short stories or 
non-fiction pieces, should be encouraged. Classroom assessment other testing 
programs could include more emphasis on this method. 

• In preparation for the 1991 Statewide Assessment, in which Reading and Literature 
will be integrated within the same tests, schools should experiment with "complete 
passages" designed to measure both types of goals (see the Language Arts Common 
Curriculum Goals). This would include "appreciation of literature" as well as 
cognitive skills. 
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You are about to read a story about something that happened in China in 400 A.D. Before 
you read the story, look at the list below. This list was created as a map of the 
background of the story. It will help you think about what might be in a story of ancient 
China. Study the list and then answer the questions (1 to 3) that follow. 



LIFE IN CHINA IN 400 A.D. 



PEOPLE 



RULERS 



Farmers 



War-like 



Suspicious of 
foreigners 



Absolute power 
Educated 



Total co»^*^rol 



Family extremely 
important 



Loyal to Emperor 
Appreciated beauty 
Saving face 



Revered 



Dy people 



Aristocracy 



Maker of rules 
and laws 



Power of life 
and death 



CULTURE/BELIEFS 



Live in harmony 
with nature 



Charity, kindness, 
hard work courtesy 



Respect 



or elders 



(Honor people 
over you — 
parents, elderly, 
emperors) 



China at center 
of world 



Strong clajs 
system 



Lots or ritual 



Protective of 
their country 



Benevolent 



Rich, educated 
people had time to 
learn and invent 



Only rich educated 



Built The Great Wall 
to protect from 
invaders 
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Answer the following questions, using the list on the previous page 
("Life in China in 400 A.D."). 



1. Which of the following was true of ancient China? 

L Groups of people rode in airplanes. 

2. There were never any wars. 

89.7% *3. Rulers were ve'-y respected. 

4. They welcomed all visitors. 

Implied 
ELS 3.1 



2. Look at the list under the heading "Rulers." Which of the following could you add 
to the list? 

68.9% *1. Could have a person killed if they were guilty of a crime. 

2. Laws could be changed by a legislature. 

3. Elected by votes from the people. 

4. Could be forced by a supreme court to stop ruling. 

Implied 
ELS 3.1 



3. Suppose you lived in 400 A.D. China and you invented a moving ramp that would 
allow you to climb over the Great Wall of China. What m^^ht the rulers say about 
your new invention? 

1. "Great! That will help our country." 

87.6% *2. "Bad! That could be used by foreign invaders." 

3. "Good! Our people need ways to cross the border." 

4. "That will help us to stay in harmony with nature." 

Implied 
ELS 3.1 



ERLC 



41 



(Go on to the Next Page) 
page 3 



Form 4 



PART 2 



Now, read the story and answer the questions that follow. 



THE FLYING MACHINE 
by Ray Bradbury 



In the year A.D. 400, Ac Emperor Yuan held 
his thnmc by die Great WaU of Oiina, and the 
land was green with rain, readying itself 
tow^ the harvest, at peace, the people in his 
dominion neither too happy nor too sad 

Early on die morning of the first day die 
first week erf die second mondi of die new 
ynr, die Enq)cror Yuan was sipping tea and 
fanning himself against a warm toe«e when a 
senrant ran across die scarlet and blue garden 
tiles, calling, "Oh, Enq)eror, a miracle!" 

"Yes " said die Emperor, "die air is sweet this 
morning." 

"No, no, a miracle!" said die servant, bowing 
quickly, 

"And this tea is good in my mouth, surely that 
is a miracle." 

"No, no, Your Excellency." 

"Let me guess dien-die sun has risen and a 
new day is upon us. Or die sea is blue. That 
now is die finest of all miracles." 

"Excellency, a man is flying!" 

••What?" The Emperor stopped his fan. 

"I saw him in die air, a man flyuig widi 
wings. I heard a voice call out of die sky, and 
when I looked up, dicre he was, a dragon in 
the heavens with a man in its moudi, a dragon 
of paper and bamboo, colored like the sun and 
die grass." 

"It is early," said die Emperor, "and you have 



just wakened from a dream." 

"It is early, but I have seen what I have seen! 
Come, and you will see it too." 

||Sit down widi me here," said die Emperor. 
"E)rink son>e tea. It must be a strange thing, if 
it is true, to see a man fly. You must have 
time to diink of it, even as I must have time to 
prepare myself for die sight" 

They drank tea. 

"Please," said die servant at last, "or he will 
be gone." 

The Enq)eror rose dioughtfiilly. "Now you 
may show me what you have seen," 

They walked into a garden, across a meadow 
of grass, over a small bridge, dirough a grove 
of trees, and up a tiiny hill 

"There!" said die servant 

The En5>en>r looked into die sky. 

And in die sky, laughing so high diat you 
could hardly hear him laugh, was a man; and 
the man was clodied in bri^t papers and 
reeds id make wings and a beautiful yellow 
tail, ?Ad he was soaring all about like die larg- 
est bird in a universe of bnds, lite a new 
dragon in a land of ancient dragons. 

The nian called down to diem from high in die 
cool winds of morning, "I fly, I fly!" 

The servant waved to him. "Yes, yes!" 

The Emperor Yuan did not move. Instead he 
looked at die Great Wall of China now taking 
shape out of die farthest mist in die green 
hills, diat splendid snake of stones which 
wridicd widi majesty across die entire land. 
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That vromicrful waU which had i»t>tect^ 
for a dnKless time fiom enen^ 1k2D^ 
picserved peace tor years wiAout number* 
He saw the town, nestled to itself by a river 
and a road and a hill, beginning to waken. 

"Tell me,- he said to his scrwmt, "has anyone 
else seen this flying man?" 

"I am the only one, Excellency," said the ser- 
vant, smiling at the sky, waving. 

The Emperor watched the heavens another 
minute and then said, "Call him down to me." 

"Ho, come down, come down! The Emperor 
wishes to sec you!" called the servant, hands 
cupped to his shoutiug mouth. 

The Emperor glanced in all dircctiOTS while 
the flying mzn soared down die morning 
wind. He saw a farmer, early in his fields, 
watching the sky, and he noted whers the 
farmer stood* 

The flying man alit with a rusde of paper and a 
creak of bamboo reeds. He came proudly to 
the Emperor, clumsy in his rig, at last bowing 
before die old man. 

"What have you done?" demanded the 
Emperor. 

"I have just told you!" cried the flier. 

•Tou have told me nothing at alL" The 
Emperw reached out a thin hand to touch the 
pretty paper and the birdlike keel of the appa- 
ratus. It smelled cool, of the wind. 

•Ts it not beautiful. Excellency?" 

•Tes, too beautiful." 

•It is the only one in the world!" smiled the 
man. "And I am the inventor." 

•The only one in the world?" 

"I swear it!" 



"Who else knows of this?" 

"No one, not even my wife, who would think 
me mad witii the sun. She tiiought I was 
making a kite. I rose in die night and walked 
to the cliffs far away. And when the morning 
breezes blew and die sun rose, I gathered my 
courage, Excellency, and leaped from the 
cliff. I flew. But my wife does not know of 
it" 

"Well for her, dien," said the Emperor. 
"Come along." 

They walked back to the great house. The sun 
was full in the sky now, and the smell of the 
grass was refreshing. The Emperor, the ser- 
vant, and the flier paused within die huge 
garden. 

The Emperor clapped his hands. "Ho, 
guards!" 

The guards came running. 

"Hold diis man." 

The guards seized the flier. 

"Call the executioner," said the Emperor. 

"What's tins!" cried the flier, bewildered. 
"What have Tdone?" He began to weep, so 
that the beautiful paper apparatus rustied. 

"Here is the man who has made a certain 
macltine," said the Emperor, "and yet asks us 
what he has created. He does not know him- 
self. It is only necessary that he create, with- 
out knowing why he has done so, or what this 
thing will do." 

The executioner came running with a sharp 
silver ax. He stood with his naked, large- 
muscled arms ready, his face covered with a 
serene white mask. 

"One moment," said the Emperor. H-^ mmed 
to a nearby table upon which sat a machine 
diat he himself had created The Emperor 
took a tiny golden key from his own aeck. 
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He fitted this key to the tiny, delicate machine 
and wound it up. Then he set the machine 
going. 

The machine was a garden of metal and 
jewels. Se't in nation, birds sang in tiny metal 
trees, wolves waited through miniature 
forests, and tiny people ran in and out of sun 
and shadow, fanning themselves with mmia- 
ture fans, lisi ^ning to die tiny emerald birds, 
and standing by impossibly small but tinkling 
fountains. 

"Is it not beautiful?" said the Emperor. "If 
you asked me what I have done here, I could 
answer you well. I have made birds sing, I 
have made forests murmur, I have set people 
to wailing in this woodland, enjoying the 
leaves and shadows and songs. That is what I 
have done." 

"But, oh. Emperor!" pleaded die flier, on his 
knees, die tears pouring down his face. "I 
have done a similar thing! I have found 
beauty. I have flown on die morning wind, I 
have looked down on all die sleeping houses 
and gardens. I have smellcd the sea and even 
seen it, beyond die hills, from my high place. 
And I have soared like a bird; oh, I cannot say 
how beautiful it is up diere, in die sky, widi 
the wind about me, die wind blowing me here 
like a feadier, diere like a fan, die way die sky 
smells in die rooming! And how free one 
feels! That is beautiful, Emperor, diat is beau- 
tiful, too!" 

"Yes," said die Emperor sadly, "I know it 
must be true. For I felt my heart move with 
you in die air and I wondered: What is it like? 
How does it feel? How do die distant pools 
look from so high? And how my houses and 
servants? Like ants? And how die distant 
towns not yet awake?" 

'Then spare roe!" 

"But diere arc times," said die Emperor, more 
sadly still, "when one must lose a littie beauty 
if one is to keep what litde beauty one aheady 
has. I do not fear you, yourself, but I fear 
another man." 



"Whatman?" 

"Some odier man who, seeing you, will build 
a tiling of bright papers and bamboo like diis. 
But the other man will have an evil face and an 
evil heart, and die beauty will be gone. It is 
diis man I fear." 

"Why? Why?" 

[•Who is to say diat someday just such a man, 
in just such an apparatus of paper and reed, 
might not fly in die sky and drop huge stones 
upon die Great Wall of China?" said die 
Emperor. 

No one moved or said a word. 

"Off widi his head," said die Emperor. 

The executioner whirled his silver ax. 

"Bum the kite and die inventor's body and 
bury dieir ashes togedier," said die Emperor. 

The servants retreated to obey. 

The Emperor turned to his handservant, who 
had seen die man flying. "Hold your tongue. 
It was all a dream, a most sorrowful and beau- 
tiful dream. And diat fanner in die distant 
field who saw, tell him it would pay him to 
consider it only a vision. If ever die word 
passes around, you and die farmer die widiin 
die hour." 

"You arc merciful. Emperor." 

"No, not merciful," said die old man. Beyond 
die garden wall he saw die guards burning die 
beautiful machine of paper and reeds diat 
smelled of die morning wind He saw the 
dark smoke climb into die sky. "No, only 
very much bewildered and afraid." He saw 
die guards digging a tiny pit wherein to bury 
die ashes. "What is die life of one man 
against diose of a million odiers? I must take 
50!agg from diat diought" 

He took die key from its chain about his necK 
and once more wound up die beautiful minia- 
ture garden. He stood looking out across die 
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land at the Gitat Wall, the peaceful town, the 
green fields, the rivers and sftpcams. He 
sighed The tiny garden whirred its hidden 
aiKi delicate machinery and set itself in nxrtion; 
tiny people walked in forests, tiny foxes loped 
through sun-speckled glades in beautiful shin- 



Re fer to the story to answer questions 4-24. 



in 3 pelts, and among the tiny trees flew little 
bits of liigh song and bright blue and yellow 
color flying, flying, flying in small sky. 

"Oh," said the Emperor, closing' his eyes, 
"look at the birds, look at the birds!" 



4. In this story, apparatus means 

1. a set of flapping wings. 

2. a newly created dragon. 

3. a large man-made bird. 
53.8% *4. a piece of equipment. 

Word Meanings 
ELS 1.2a 



5. In this story, hordes means 

1 . insects flying around in a circle. 

2. large groups of wild animals. 

3. dangerous spies. 

57.6% *4. very large group of people. 

Word Meanings 
ELS 1.2a 



6. In this story, dominion means 

1 . a land of green grass. 

2. emperor's castle. 
56.3% *3. territory or land. 

4. people who live in China. 

Word Meanings 
ELS 1.2a 
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7. In this story, solace means 

1. grief. 

35.7% *2. comfort. 

3. pity. 

4. revenge. 

Word Meanings 
ELS 1.2a 



8. What was called "that splendid snake of stones" in the story? 

1. the emperor's invention 

2. the servant's whip 

3. the flying apparatus 
84.9% *4. the Great Wall of China 

Literal 
ELS 2. Id 



9. Which of the following best explains what happened in the story? 

1. The Emperor felt man was not meant to fly and so he ordered the man 
killed. 

2. A cruel Emperor has an inventor beheaded so he can enjoy the birds. 
77.9% *3. An inventor is executed by an Emperor who worried about the future 

security of the wall. 
4. A man is executed because a jealous Emperor doesn't like his invention. 

Literal 
ELS 2. Id 



10. Which of the following happens ^nmediately after the Emperor sees the flying 
man? 

1. He shows the flying man his own invention. 
73.9% *2. He checks to see who else has seen the invention. 

3. He orders the executioner to behead the man. 

4. He asks the man to explain his invention. 

Literal 
ELS 2.1c 
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11, Which is the correct sequence of events in the story? 

L The Emperor talks to the flying man, burns the fljing apparatus, and has 
the man beheaded. 

66.3% *2. After being told to look in the sky, the Emperor sees a man flying, talks 
to him and has him executed. 

3. A servant tells the Emperor of the flying machine, the Emperor destroys 
the machine and then has the man beheaded. 

4. The Emperor shows the flying man his invention, calls for the executioner 
and has the man beheaded. 

Literal 
ELS 2.1c 



12. You can conclude from the story that the Emperor 

81.0% *1. felt the flying man's invention could be used for evil purposes. 

2. was jealous of any other man's invention. 

3. executed people for no apparent reason. 

4. did not like any apparatus made of bamboo. 

Implied 
ELS 3.1b 



13, What can you conclude from the story about the rulers of China in 400 A.D.? 

81.8% *1. They had control over life and death. 

2. They loved only beautiful, miniature machines. 

3. They had courts and juries to make decisions. 

4. They felt safe and secure from their enemies. 

Implied 
ELS 3. If 



14, When the Emperor asks the flier, •'What have you done?" he really means 

82,8% *1. "Do you realize the danger this could bring?" 

2. "What can you see from the sky?" 

3. "How did you go about building the machine?" 

4. "What did you use to make this beautiful apparatus?" 

Implied 
ELS 3. If 
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IS. To the Emperor, the Great Wall of China represents 

85.0% *1. security. 

2. beauty. 

3. danger. 

4. happiness. 

Implied 
ELS 3.1 



16. Which best characterizes the Emperor? 

1 . kind and curious 

78.7% *2. wise but cautious 

3. cruel and uncaring 

4. stupid but ^riendly 

Implied 
ELS 3. If 



17. Why was the life of the inventor's wife saved? 

1. The Emperor didn't know where to find her. 

2. She thought her husband was "mad". 

81.3% *3. She did not know that he built a flying machine. 

4. She was not present at the time the inventor landed in the field. 

Implied 
ELS 3. If 



18. Which of the following quotations best illustrates the main message of the stoiy? 

1. "Oh, Emperor, Emperor, a miracle!" 
69.9% *2. "But there are times when one must lose a little beauty if one is to keep 
what little beauty one already has." 

3. "Here is the man who has made a certain machine and yet asks us what he 
has created." 

4. "I have made the birds sing, I have made the forests murmur." 

Implied 
ELS 3.1c 
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19. Why did the author say that the Great Wall of China "writhed with majesty across 
the entire land?" 

1. It was broken in several places. 

2. It covered the entire landscape. 
64.6% *3. It twisted over the land like a snake. 

4. He was describing a type of large snake found all over China. 

Implied 
ELS 3. If 



20. How is the point -of-view of the Emperor and the inventor the same? 

71.6% *1. Both search for beauty. 

2. Both are afraid of change. 

3. Both want to visit distant lands. 

4. Both like power and control of other people. 

Evaluative 
ELS 6.2b 



21. According to the Emperor, what is the difference between his invention and the 
man's flying apparatus? 

1. The man's was dangerous; the Emperor's more creative. 

65.5% *2. The man's was chreatening; the Emperor's close to nature. 

3. The man's was ugly; :he Emperor's beautiful. 

4. The man's was bamboo; the Emperor's paper. 

Evaluative 
ELS 6.2 
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22. Based on the story how do you think the Emperor will react the next time he sees a 
new invention. 

1. Accept it if it is a piece of art. 

2. Ask the inventor to make him one. 
73.0% *3. Question it's use and function. 

4. Immediately have the man beheaded. 

Implied 
ELS 3.1e 
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23, Which of the Emperor's statements is a fact? 

1. "The air is sweet this morning." 

2. "It is only necessary that he create, without knowing why . . 
70.3% *3. "The sun has risen and a new day is up>on us." 

4. "It was all a dream, a most sorrowful and beautiful dream." 

Literal 
ELS 3.1a 



24, What is the best way to describe the Emperor's statement, "I do not fear you, 
yourself, but I fear another man. The other man will have an evil face and an evil 
heart." 

1. a fact or finding 

2. a persuasion technique 
68.4% *3. a conclusion or prediction 

4. an impossible idea 

Evaluative 
ELS 6.4 
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